Skip to content

Add aggregator CLI commands for configuration parameters - #3438

Open
turmelclem wants to merge 15 commits into
mainfrom
ctl/3392-add-aggregator-CLI-command-for-configuration-parameters
Open

Add aggregator CLI commands for configuration parameters#3438
turmelclem wants to merge 15 commits into
mainfrom
ctl/3392-add-aggregator-CLI-command-for-configuration-parameters

Conversation

@turmelclem

@turmelclem turmelclem commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Content

aggregator cli :

  • export command
    • look for markers on chain and export then into JSON format (if it exists, otherwise it export a default configuration)
  • import command generating datum file from a JSON file
    • verifying user JSON file consistency (non zero values, configurations and enabled signed entity type consistency)
    • verifying configuration to import against configuration on chain TODO
    • checking datum that not exceed 10kb (since a full cardano transaction is 16kb) TODO
    • writing a JSON HEX datum file at given path

protocol-config :

  • a adapter trait that is used by aggregagtor cli to retrieve information about protocol configuration markers
    • with the intelligence to retrieve the configuration of the nearest known epoch
  • cbor encoding/decoding of marker configuration

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • All check jobs of the CI have succeeded
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested
  • Documentation
    • Update README file (if relevant)
    • Update documentation website (if relevant)
    • Add dev blog post (if relevant)
    • Add ADR blog post or Dev ADR entry (if relevant)
    • No new TODOs introduced

Issue(s)

this PR closes #3392

@turmelclem turmelclem self-assigned this Jul 27, 2026
Comment thread internal/mithril-protocol-config/src/adapters/cardano_chain.rs Fixed
Comment thread internal/mithril-protocol-config/src/adapters/cardano_chain.rs Fixed
Comment thread internal/mithril-protocol-config/src/adapters/cardano_chain.rs Fixed
Comment thread internal/mithril-protocol-config/src/adapters/cardano_chain.rs Fixed

@github-advanced-security github-advanced-security AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clippy found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

Test Results

     5 files  ± 0     209 suites  ±0   50m 22s ⏱️ - 2h 24m 50s
 3 354 tests  - 52   3 354 ✅  - 52  0 💤 ±0  0 ❌ ±0 
11 167 runs   -  8  11 167 ✅  -  8  0 💤 ±0  0 ❌ ±0 

Results for commit 0f67a96. ± Comparison against base commit bf1c9b3.

This pull request removes 76 and adds 24 tests. Note that renamed tests count towards both.
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::index_out_of_bounds
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::index_too_large_for_circuit_range
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::indices_not_increasing
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_merkle_path_mismatch
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_swap_keep_merkle_path
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_wrong_verification_key
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_corrupt_sibling
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_flip_position
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_length_long
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_length_short
…
mithril-aggregator ‑ commands::protocol_configuration_command::tests::export_subcommand_parses_flag
mithril-aggregator ‑ commands::protocol_configuration_command::tests::import_subcommand_parses_flag
mithril-aggregator ‑ commands::protocol_configuration_command::tests::to_vec_human_readable_protocol_configuration_converts_markers_to_human_readable_list
mithril-aggregator ‑ commands::protocol_configuration_command::tests::verify_protocol_configurations::shoud_throw_error_if_enabled_entity_types_contains_cardano_blocks_transactions_without_configuration
mithril-aggregator ‑ commands::protocol_configuration_command::tests::verify_protocol_configurations::shoud_throw_error_if_enabled_entity_types_contains_cardano_transactions_without_configuration
mithril-aggregator ‑ commands::protocol_configuration_command::tests::verify_protocol_configurations::should_throw_error_with_protocol_parameter_at_zero
mithril-aggregator ‑ tools::protocol_configuration::tests::from_human_readable_protocol_configuration_converts_to_protocol_configuration_for_epoch
mithril-aggregator ‑ tools::protocol_configuration::tests::generate_tx_datum_ok
mithril-aggregator ‑ tools::protocol_configuration::tests::verify_configurations_against_chain::full_offset_window_have_to_be_repeated_if_it_have_different_configuration
mithril-aggregator ‑ tools::protocol_configuration::tests::verify_configurations_against_chain::ko_because_last_known_on_chain_configuration_b_for_offset_window_is_not_repeated
…

♻️ This comment has been updated with latest results.

Comment thread internal/mithril-protocol-config/src/cardano_chain/mod.rs Fixed
Comment thread internal/mithril-protocol-config/src/cardano_chain/mod.rs Fixed
Comment thread internal/mithril-protocol-config/src/cardano_chain/mod.rs Fixed
Comment thread internal/mithril-protocol-config/src/lib.rs Fixed
Comment thread internal/mithril-protocol-config/src/cardano_chain/mod.rs Fixed
Comment thread internal/mithril-protocol-config/src/cardano_chain/mod.rs Fixed
Comment thread internal/mithril-protocol-config/src/cardano_chain/mod.rs Fixed
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Fixed
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Fixed
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Fixed
Comment thread mithril-aggregator/src/dependency_injection/containers/serve.rs Fixed
Comment thread mithril-aggregator/src/dependency_injection/containers/serve.rs Fixed
@turmelclem
turmelclem force-pushed the ctl/3392-add-aggregator-CLI-command-for-configuration-parameters branch 3 times, most recently from ddfd41b to 10aa1ef Compare July 30, 2026 12:38
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Fixed
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Fixed
@turmelclem
turmelclem marked this pull request as ready for review July 31, 2026 12:05
@turmelclem
turmelclem force-pushed the ctl/3392-add-aggregator-CLI-command-for-configuration-parameters branch from b51783c to 39da965 Compare July 31, 2026 12:11
@turmelclem
turmelclem force-pushed the ctl/3392-add-aggregator-CLI-command-for-configuration-parameters branch 5 times, most recently from 2d65553 to ccb24d2 Compare July 31, 2026 13:10
@jpraynaud
jpraynaud requested a review from Copilot July 31, 2026 13:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces new aggregator CLI commands to export/import protocol configuration markers, along with a new mithril-protocol-config reader interface/model and Cardano-chain message/payload encoding support to make protocol configuration markers portable and verifiable.

Changes:

  • Add protocol-configuration export-markers / import-markers CLI commands (JSON export + datum generation/import validation).
  • Add protocol-configuration “tools” and dependency injection wiring to read on-chain markers and generate signed datums.
  • Add mithril-protocol-config models/messages (CBOR encoding) + payload signing structure + test doubles.

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
mithril-stm/Cargo.toml Switch ciborium to workspace dependency for consistent versioning.
mithril-common/src/entities/type_alias.rs Add hex-encoded secret key type alias for protocol configuration markers.
mithril-common/src/crypto_helper/ed25519_alias.rs Add ED25519 type aliases for protocol configuration markers signing/verifying.
mithril-common/Cargo.toml Switch ciborium to workspace dependency.
mithril-aggregator/src/tools/protocol_configuration.rs New tool: verify imported configs against chain window + build signed datum + datum size check.
mithril-aggregator/src/tools/mod.rs Register/export ProtocolConfigurationTools.
mithril-aggregator/src/test/double/dummies.rs Add Dummy impl for HumanReadableProtocolConfiguration for tests.
mithril-aggregator/src/dependency_injection/containers/protocol_configuration.rs New DI container for protocol-configuration commands.
mithril-aggregator/src/dependency_injection/containers/mod.rs Export the new protocol-configuration DI container.
mithril-aggregator/src/dependency_injection/builder/support/compatibility.rs Add construction + getter for a ProtocolConfigurationMarkersReader (prod vs dummy).
mithril-aggregator/src/dependency_injection/builder/mod.rs Add stored dependency + container creation method for protocol-configuration commands.
mithril-aggregator/src/configuration.rs Add ProtocolConfigurationReaderParameters + ConfigurationSource accessor for reader params.
mithril-aggregator/src/commands/protocol_configuration_command.rs Implement CLI subcommands, JSON IO, consistency checks, datum generation.
mithril-aggregator/src/commands/mod.rs Wire new ProtocolConfiguration command into main CLI.
internal/mithril-protocol-config/src/test/double/mod.rs Export new dummy markers reader from test doubles.
internal/mithril-protocol-config/src/test/double/dummy_markers_reader.rs Add dummy ProtocolConfigurationMarkersReader for test environments.
internal/mithril-protocol-config/src/test/double/dummies.rs Add dummies for protocol-configuration marker models/messages.
internal/mithril-protocol-config/src/model.rs Add protocol configuration marker model + ConfigurationComputerFromMarkers helper with fallback lookup.
internal/mithril-protocol-config/src/lib.rs Export cardano_chain module publicly.
internal/mithril-protocol-config/src/interface.rs Add ProtocolConfigurationMarkersReader trait.
internal/mithril-protocol-config/src/cardano_chain/protocol_configuration_reader.rs Add Cardano-chain reader type for markers (currently stubbed).
internal/mithril-protocol-config/src/cardano_chain/payload.rs Add payload structs and signing helper for protocol configuration markers.
internal/mithril-protocol-config/src/cardano_chain/mod.rs Add Cardano-chain module exports for message/payload/reader.
internal/mithril-protocol-config/src/cardano_chain/message.rs Add CBOR message/codec for protocol configuration markers.
internal/mithril-protocol-config/Cargo.toml Add dependencies needed for CBOR/hex/serde + chain reader integration.
Cargo.toml Add workspace ciborium dependency.
Cargo.lock Lockfile updates for new deps/crate wiring.
Suppressed comments (1)

mithril-aggregator/src/commands/protocol_configuration_command.rs:345

  • User-facing CLI output contains spelling/grammar issues ("Sucessfuly write"), which makes the command output look unpolished.
        println!(
            "Sucessfuly write Tx datum file at {}",
            self.target_path.to_string_lossy()
        );

Comment thread mithril-aggregator/src/tools/protocol_configuration.rs Outdated
Comment thread mithril-aggregator/src/tools/protocol_configuration.rs
Comment thread mithril-aggregator/src/tools/protocol_configuration.rs
Comment thread mithril-aggregator/src/tools/protocol_configuration.rs Outdated
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Outdated
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs
Comment thread mithril-aggregator/src/commands/protocol_configuration_command.rs Outdated
Comment thread internal/mithril-protocol-config/src/cardano_chain/message.rs Outdated
Comment thread internal/mithril-protocol-config/src/model.rs
@turmelclem
turmelclem force-pushed the ctl/3392-add-aggregator-CLI-command-for-configuration-parameters branch from ccb24d2 to 95471fe Compare July 31, 2026 15:01
…rsion, signature and generating tx datum for protocol configuration command
…ct to avoid missing signing when generating protocol config tx datum
@turmelclem
turmelclem force-pushed the ctl/3392-add-aggregator-CLI-command-for-configuration-parameters branch 2 times, most recently from ad5d80a to aba2fbb Compare July 31, 2026 15:28
@turmelclem
turmelclem force-pushed the ctl/3392-add-aggregator-CLI-command-for-configuration-parameters branch from aba2fbb to 0f67a96 Compare July 31, 2026 15:54
@turmelclem
turmelclem temporarily deployed to testing-preview July 31, 2026 16:09 — with GitHub Actions Inactive
@turmelclem
turmelclem temporarily deployed to testing-2-preview July 31, 2026 16:09 — with GitHub Actions Inactive
@jpraynaud
jpraynaud requested a review from Copilot August 3, 2026 16:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (10)

internal/mithril-protocol-config/src/cardano_chain/protocol_configuration_reader.rs:45

  • CardanoChainProtocolConfigurationMarkersReader::read is wired for ExecutionEnvironment::Production (via DependenciesBuilder::build_protocol_configuration_reader) but currently always returns an "not implemented yet" error. This means the new production CLI commands will fail at runtime, and it contradicts the PR/issue requirement to read on-chain markers.
    async fn read(&self) -> StdResult<ConfigurationComputerFromMarkers> {
        //read payload
        // to ProtocolConfigurationForEpochMessage
        // to ProtocolConfigurationForEpoch
        // build ConfigurationComputerFromMarkers with ProtocolConfigurationForEpoch
        Err(anyhow::anyhow!(
            "CardanoChainProtocolConfigurationMarkersReader::read is not implemented yet"
        ))

mithril-aggregator/src/commands/protocol_configuration_command.rs:345

  • Typo in user-facing output: "Sucessfully" should be "Successfully".
        println!(
            "Sucessfully wrote Tx datum file at {}",
            self.target_path.to_string_lossy()
        );

mithril-aggregator/src/commands/protocol_configuration_command.rs:310

  • fs::read_to_string is stored as a Result and then immediately ?-unwrapped inside serde_json::from_str. This is harder to read than unwrapping once and passing the String to serde.
        let json_protocol_configurations = fs::read_to_string(&self.import_path);

        //2 - Parse the json into a protocol configuration list using serde_json
        println!("Json parsing ...");
        let protocol_configurations: Vec<HumanReadableProtocolConfiguration> =
            serde_json::from_str(&json_protocol_configurations?)?;

mithril-aggregator/src/tools/protocol_configuration.rs:157

  • This tool code uses println! for reporting datum size even though the rest of the module uses slog logging. Using the logger keeps output consistent and configurable (and avoids mixing stdout/stderr in services).
        println!("Datum size: {:.2} KB", size_kb);

mithril-aggregator/src/tools/protocol_configuration.rs:31

  • Typo in error message: "the same has configuration" should be "the same as configuration".
    #[error("Configuration to import for {0:?} is not the same has configuration on chain")]
    NotSameConfigurationForEpoch(Epoch),

mithril-aggregator/src/commands/protocol_configuration_command.rs:227

  • Typo in user-facing output: "Sucessfully" should be "Successfully".

This issue also appears on line 342 of the same file.

        println!(
            "Sucessfully wrote JSON protocol configurations file at {}",
            self.target_path.to_string_lossy()
        );

mithril-aggregator/src/tools/protocol_configuration.rs:83

  • get_on_chain_configurations(self) consumes the whole ProtocolConfigurationTools, which is surprising for a getter and prevents calling other methods afterward. Consider taking &self and returning a clone (or returning a reference) so the tools instance remains usable.
    pub fn get_on_chain_configurations(self) -> ConfigurationComputerFromMarkers {
        self.configuration.on_chain_configurations
    }

mithril-aggregator/src/configuration.rs:204

  • This newly added ConfigurationSource::protocol_configuration_reader_parameters default implementation uses panic!. Since this method is used to build production dependencies, leaving it as a panicking default increases the risk of runtime crashes if an implementation forgets to override it. Prefer making it a required method (no default) or returning a Result with a proper error.
    /// Protocol configuration reader adapter parameters
    fn protocol_configuration_reader_parameters(&self) -> ProtocolConfigurationReaderParameters {
        panic!("protocol_configuration_reader_parameters is not implemented.");
    }

internal/mithril-protocol-config/src/cardano_chain/payload.rs:93

  • ProtocolConfigurationMarkersPayload::sign maps serialization errors from message_to_bytes() into CreateSignature, which misclassifies the failure (and discards the more specific SerializeMessage variant). The serialization error should be propagated as-is before signing.
        let signature =
            signer.sign(&self.message_to_bytes().map_err(|e| {
                ProtocolConfigurationMarkersPayloadError::CreateSignature(e.into())
            })?);

mithril-aggregator/src/commands/protocol_configuration_command.rs:3

  • Map and Value are imported from config but never used in this file; this will trigger unused-import warnings (CI should be warning-free).

This issue also appears on line 305 of the same file.

use config::{ConfigBuilder, Map, Value, builder::DefaultState};

#[error("could not verify signature")]
VerifySignature(#[source] StdError),

/// Error raised when the signing the markers

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not very clear.

use crate::{dependency_injection::DependenciesBuilder, tools::ProtocolConfigurationTools};

#[derive(Debug, Error)]
pub enum UserConfImportVerificationError {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub enum UserConfImportVerificationError {
pub enum InputConfigurationImportVerificationError {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All pub struct should have a doc comment. In particular, the one associated to the command is used by clap to create the help of the CLI:

Commands:
  genesis                 Genesis tools
  era                     Era tools
  serve                   Server runtime mode
  tools                   List of tools to upkeep the aggregator
  database                Database tools
  protocol-configuration  
  help                    Print this message or the help of the given subcommand(s)

#[error("Protocol parameters must be non-zero: {0:?}")]
ZeroValueProtocolParameters(ProtocolParameters),

#[error("enabled_signed_entity_types contains '{0:?}' without any associated configuration")]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message is not clear.

root_logger: Logger,
config_builder: ConfigBuilder<DefaultState>,
) -> StdResult<()> {
// 0 conf & dependencies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All comments in the execute function of this command and the following one could follow the same pattern: // X: comment.

Comment on lines +427 to +428
on_chain_markers.insert(Epoch(38), fake_configuration('A')); //conf outside offset window
on_chain_markers.insert(Epoch(44), fake_configuration('B')); //conf inside offset window

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO these comments can be removed as they don't bring a lot (and they lack correct formatting).

}

/// Encode this payload to a json hex string
pub fn to_json_hex(&self) -> StdResult<String> {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably miss some golden tests here?

};

/// The cbor representation of a [ProtocolConfigurationForEpochMessage]
pub type CborProtocolConfigurationForEpochMessage = String;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more CborHex tha Cbor.

Ok(hex::encode(self.to_cbor_bytes()?))
}

/// Deserialize a type `T: Serialize + DeserializeOwned` from CBOR bytes representation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment and the one below probably need to be adjusted.

}

#[cfg(test)]
mod tests {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Golden tests are also welcome here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement aggregator CLI commands for configuration parameters

4 participants